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AUTOMATED CENTERLINE DETECTION ALGORITHM FOR 



COLON-LIKE 3D SURFACES 



[0001] This invention relates to virtual colonoscopy and includes processing computer 
tomographic image slices of the patient. The disclosure assumes the procurement of a three 
5 dimensional image of the colon like surface of vertex points distributed along an elongate 
major axis, each vertex point having a discrete point identifier and three dimensional position 
information. In this disclosure, an accurate ring profile of the colon like surface can be 
generated. Utilizing the accurate ring profile, reconstruct the 3D surface of the colon, serially 
view the colon cross-section for anomalies such as polyps and thus provide both non-invasive 
10 colon screening together with the positional information needed for follow up conventional 
colonoscopy. 



[0002] Virtual Colonoscopy is a relatively new field in medicine and consists in processing 
the computer tomographic (CT) image planes or "slices*' of the patient rather than the 

15 traditional and invasive colonoscopic inspection. By segmenting the original parallel CT 
slices, it is possible to reconstruct the 3D surface of the colon and thus provide the 
information needed for Colonoscopy. Such a process is disclosed in Summers et al. US 
Patent 6,246,748 issued June 12, 2001 and entitled Method for Segmenting Medical Images 
and Detecting Surface Anomalies in Anatomical Structures. This disclosure assumes as a 

20 starting point that an image similar to that produced by Summers et al *748 is produced. 
[00031 Assuming the presence of such an image, further processing of the image such as 
detecting the centerline of the colon is still a challenging problem. The following articles are 
representative of relevant image processing techniques that can be used to address this 
problem: 

25 [0004] Gabriel Taubin, "Curve and Surface Smoothing without Shrinkage", IEEE (1995); 
[0005] Ingmar Bitter, et al., "Penalized-Distance Volumetric Skeleton Algorithm", IEEE 
Transactions on Visualization and Computer Graphics, Vol.7, No. 3, July — September 



30 efficient method based on 3D skeletons", J Comput Assist Tomogr, 23(5): 786-94 (1999); 
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2001; 



[0006] Ge, Y., D. R. Stelts, et al., "Computing the centerline of a colon; a robust and 
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[0007] Jie Wang, Yaorong Ge, "An Optimization problem in Virtual Colonoscopy", 
Theoretical Computer Science, Elsevier Science B. V. 9 pp 203-216 (1998); 
[0008] Yaorong Ge, et aL, "Computing the Centerline of a Colon: a Robust and Efficient 
Method Based on 3D Skeletons", Journal of Computer Assisted Tomography, 23(5):786-794 ^ 
5 (1999); 

[0009] Rui Chiou, et aL, "An Interactive Fly-Path Planning Using Potential Fields and Cell 
Decomposition for Virtual Endoscopy", IEEE Transactions on Nuclear Science, Vol 46, No. 
4, August 1999; 

[0010] David Paik, et aL, "Automated flight path planning for virtual endoscopy", 
10 American Assoc Of Physicists in Medicine (1998); 

[0011] CX. Wyatt, et aL, "Automatic Segmentation of the Colon for Virtual Colonoscopy", 
Elsevier Science (2000); 

[0012] Elisabeth McFarland, et aL, "Spiral Computed Tomographic Colonography: 
Determination of the Central Axis and Digital Unravelling of the Colon", Technical Report, 
15 Acad. Radiology, 4:367-373 (1997); 

[0013] Roel Truyen, et aL, "Efficacy of automatic path tracking in Virtual colonoscopy", 
Cars 2001 —H. U. Lemke, et aL (Editors), Elsevier Science (2001). 

[0014] C. E. Shannon, "A Mathematical Theory of Communication", Bell System Tech. J. 9 
379-423 (1948); and 

20 [0015] W. J. Schroeder, et aL, "Decimation of Triangle Meshes", Computer Graphics, 
26:26 (1992). 

[0016] The reader will understand that the colonoscope is the usual instrument used both 
for inspection and surgery of the colon. The measurements of the colon must be all related to 
the path of the colonoscope, always from the rectum. In making its penetration of the colon, 

25 the colon "gathers" the invading colonoscope into the serpentine path of the major axis of the 
colon while the colonoscope "hunts" this path within the confines of the colon. As a 
consequence of this well known and understood technique of using a colonoscope, relevant 
measurements taken with respect to the colon by non-invasive CT scans must all be related to 
penetration of a colonoscope. ' 

30 [0017] Further, scans taken by a colonoscope must be able to be compared to non-invasive 
scans taken by computer tomography to establish the efficacy of the non-invasive scans. 
Likewise, and assuming efficacy of the non-invasive scans, the findings of the non-invasive 
scans must be directly translatable to the doctor using colonoscope to be of practical value. 
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BRIEF SUMMARY OF THE INVENTION 
[0018] A virtual non-invasive colonoscopy of a colon like surface includes processing 
computer tomographic image slices of the patient. The procurement of a three dimensional 
image of the colon like surface of vertex points, each vertex point having a discrete point 
5 identifier and three dimensional position information is presumed. The three dimensional 

position information of all vertex's neighbors is averaged in a shrinking procedure to contract 
the three dimensional image proximate to a major axis of the colon-like surface, with the 
overall length of the colon along the major axis remaining substantially unchanged. Evenly 
spaced points are taken through the shrunken colon like surface and connected to form a 

10 curve. Planes are generated at intervals normal to the curve to split the shrunken colon like 
surface into image segments. By mapping these image segments back to the original image 
through their discrete vertex point identifiers, an accurate ring profile of the colon like 
surface can be generated. Utilizing the accurate ring profile, it is possible to compute an 
accurate colon centerline, serially view the colon cross-section for anomalies such as polyps 

1 5 and thus provide both non-invasive colon screening together with the positional information 
needed for follow up conventional colonoscopy. 

[0019J As of the filing of this Patent Cooperation Treaty application, an improved 
technique for modeling of the centerline of the colon includes taking the image of the 
shrunken colon and choosing and marking a first random vertex. The reader will understand 

20 that all vertices on the shrunken image of the colon are by definition very close to the colon 
centerline by virtue of the fact of the image being a shrunken image. Once this first random 
vertex is choosen, all unmarked neighbors of the random vertex of the shrunken image of the 
colon are identified and marked. The identification and marking process is sequentially 
repeated from each marked neighbor to all unmarked neighbors. When the identification and 

25 marking process propagates a predetermined distance (preferably a multiple of the maximum 
diameter of the shrunken colon, plus or minus a chosen tolerance), a second random vertex is 
designated. A third random vertex is chosen at substantially the same interval from the first 
random vertex, but in the opposite direction. Thus, starting with the first random vertex, 
marking of the vertices propagates from the first random vertex to both ends of the shrunken 

30 colon image. In this process, all unmarked neighbors of each random vertex are identified 
and marked until the process propagates the predetermined distance, a new random vertex is 
designated, and marking process continued. The marking of vertices thus proceeds from the 
first random vertex, located centrally of the colon, to spaced random vertices extending along 
the colon to the respective ends of the colon at the anal verge and the cecum. 
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[0020] The reader will understand that since we are working on an image of a shrunken 
colon, all vertices on the surface of the shrunken image must by definition be very close to 
the colon centerline. Thus, the shrunken colon surface, the centerline of the shrunken colon, 
and the centerline taken along the length of the colon are all essentially the same. They have 
the essentially the same (serpentine) three dimensional position in space along the length of 
the colon. 

[0021] An advantage of this marking system is that it eliminates the use of vectors in 
determining the centerline of the colon. The simple process of vertex identification and 
marking followed by designating vertices systematically . tracks the length of the shrunken 
image of the colon from the first random vertex to both colon ends. 
[0022] In the following disclosure, it is necessary to define terms. These terms are: 
[0023] Colon like surface: This term obviously includes the colon. It includes other 
surfaces within the human body that are elongate about what can be generally termed a major 
axis or axes. Such surfaces can also include blood vessels, biliary ducts, airways, small 
intestine, spinal canal and cord. 

[0024] Major axis or axes: When one views a body structure, such as the colon, an irregular 
enclosed but generally cylindrical elongate structure is shown. Elongation can be said to 
occur generally proximate to a major axis of the structure. This major axis is not the same as 
the ultimately computed centerline. Further, this term is used in this disclosure to express an 
intuitive concept perceived by observing the elongate colon structure. 

[0025] Vertex point: This is the discrete informational image point that together with many 
other similar image points makes up the initial image processed by this invention. Such 
vertex points each have a discrete identifier. Further, the vertex points have three dimensional 
image information, commonly in Cartesian coordinates. As will be apparent in what follows, 
this disclosure alters the positional information in a shrinking process. The shrinking process 
leaves the colon like surface length substantially unchanged along the major axis but 
drastically shrinks the width of the colon like surface. Once shrinking has occurred, the 
shrunken image is parsed and mapped back to the original image utilizing the discrete 
identifiers. 

* 

[0026] Shrunken colon like surface: This is a 3D surface, obtained from the originally 
supplied 3D-colon like surface. The shrunken colon is very thin and almost as long as the 
original colon. The shrunken colon like surface has the same number of vertices as the 
original colon and the same "structure." The vertices neighborhood relation is the same, 
indexed (triangle) strip sets are the same. 



WO 03/058553 



PCT/US02/40641 



[0027J Stated in theoretical terms, this means that, for every integer "i" and "k", if vertex 
Vi (vertex number i) has vertex Vk (vertex number k) as a neighbor in the original colon, 
there will be a vertex Vi in the shrunken colon that will have a neighbor Vk. The only 
difference will be in the three dimensional image information. In the "shrunken colon like 
5 surfaces'", the distances from the major axis to the vertex points will be almost 10 to 100 
times smaller in the shrunken colon as in original colon image. 

[0028] Rings: It will be understood that slices of the (original or shrunken) colon like 
surfaces generally are normal to the major axis of the elongate surface. These slices are 
deformed cylinders. These cylinders will be large in the original image and small in the 

10 shrunken image and composed of vertex points. We also want the axis of these cylinders to 
be parallel to (if not included in) the centerline of the original or shrunken colon. When this 
happens we also refer to such slices as "perpendicular rings." 
[0029] Centerline: This is the most useful output of our image manipulation. It 
approximates the gathered path of a colonoscope as it hunts centrally of the colon. It is close 

15 to the more mathematically elaborated concept of skeletons. We want the centerline to 

closely model the inserted profile of the colonoscope, which is semi-rigid and cannot make 
sharp bends. Intuitively the centerline of a colon-like surface should be the curve that stays 
inside the colon and never goes "too close" to the walls. 

[0030] We use the computed centerline to discriminate various colon areas that are polyp 
20 candidates in order to be able to evaluate the accuracy of the polyp detection software 

module, such as that set forth in Summers et al US Patent 6,246,748 issued June 12, 2001 
entitled Method for Segmenting Medical Images and Detecting Surface Anomalies in 
Anatomical Structures. Utilizing centerline information realized from the processed original 
image of the colon like surface, the software-detected polyps are compared with the human 
25 detected polyps to both establish efficacy of the disclosed technique and to render the 
disclosed non-invasive colonoscopy a useful tool for further colonoscope inspection and 
surgery. In most cases, comparison is based on the distance to and from the anal verge. These 
provided values are actually based on the length of the colonoscope at the moment of polyp 
detection. 

30 BRIEF DESCRIPTION OF THE DRAWINGS 

[0031] Fig. 1 is a perspective view of a typical human colon omitting related anatomy 
illustrating the exterior profile obtained by the disclosure of Summers et al. US Patent 
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6,246,748 issued June 12, 2001 and entitled Method for Segmenting Medical Images and 
Detecting Surface Anomalies in Anatomical Structures; 

[0032] Fig. 2 A is a two dimension plot of vertex points illustrated along a circular path 
illustrating shrinkage of the circle by the averaging technique of this invention; 
5 [0033] Fig. 2B is a two dimensional plot of vertex points illustrated along a linear path 
illustrating no shrinkage of the linear path by the averaging technique of this invention; 
[0034] Fig. 3 is the colon of Fig. 1 having been shrunken by "neighbors averaging" with 
about 250 iterations; 

[0035] Fig. 4 is a schematic of a three dimensional "neighbors averaging" illustrating the 
1 0 alteration of three dimensional position of the vertex points along a length of a colon having 
its local major axis oriented horizontally; 

[0036] Fig. 5 is a schematic of a three dimensional "neighbors averaging" illustrating the 
alteration of three dimensional position along a length of a colon having its local major axis 
undertaking relatively large curvature; 
1 5 [0037] Fig. 6 is a schematic of vertex points taken along a shrunken colon illustrating how 
previous selected random vertex points generate both a position or origination of a vector and 
a search volume defined about the vector to enable selection of the next random vertex point 
along the length of the shrunken colon; 

[0038] Fig. 7A is a schematic of a search sphere segment taken along a solid angle 
20 generated from a vector defined by two previously located random vertex points; 

[0039] Fig. 7B is a schematic on a reduced scale illustrating the location of the random 
vertex points along the length of the shrunken colon; 

[0040] Fig. 8 is a schematic detail illustrating the difficulty of tracking random vertex 
points where the colon-like surface being tracked undertakes a sharp bend; 
25 [0041] Fig. 9 is a schematic diagram illustrating the construction of planes normal to model 
curve, and passing through points in the curve constructed from randomly determined data 
points along the length of the shrunken colon to separate out those vertex points local to a 
ring segment of the colon; 

[0042] Fig. 10 is a schematic illustrating vertex points from between two planes taken in 
30 Fig. 9 being mapped to the original image of the colon; 

[0043] Fig. 1 1 A display a ideal ring when the colon is ideal - a twisted cylinder and 1 IB 
illustrates the real situation when the rings are not small cylinders. We also suggest another 
technique, simpler but different than averaging technique that is used in our implementation, 
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for determining the centerline of the colon in the real situation: we can compute the centerline 
point using the center of the bounding box of the rings; 

[0044] Fig. 12 illustrates a coordinate Cartesian coordinate convention superimposed upon 
a typical human colon; and, 

[00451 Fig. 13 is a schematic illustrating a colon stretched along a linear path. 
[0046] Fig. 14 is a schematic of a thinned colon image illustrating the selection of a random 
vertex point VO illustrating successive identification and marking of the neighboring vertex 
points with propagation occurring generally from the left to the right of the illustration; and, 
[0047J Fig. 15 is a schematic of a thinned colon image illustrating designation of a random 
vertex points from an initially selected first random of vertex to both colon ends. 

DETAILED DESCRIPTION OF THE INVENTION 
[0048] Referring to Fig. 1, we assume the acquisition of a 3-D image of the colon like 
surface such as that disclosed in Summers et al. US Patent 6,246,748 issued June 12, 2001 
and entitled Method for Segmenting Medical Images and Detecting Surface Anomalies in 
Anatomical Structures. We are using the results of the module which process the CT slices 
and computes the 3D surface of the colon in accordance with that disclosure. In this view, the 
anal verge 14 and the cecum 16 can be seen at either end of the colon. 
[0049] Having acquired such an image, our problem is: given the 3D surface of the colon 
we want to find an (ordered) set of 3D points which define the colon s centerline. 
[0050] The method involves basic steps: 

[0051] 1) Compute a shrunken version of the colon. Basically the distances between 
vertices are iteratively averaged, the result being a shrunken colon. The shrunken colon is 
very thin and almost as long as the original colon. 

[0052] 2) Model the shrunken colon by an ordered group of 3D points along the major 
axis of the colon. We take equally distanced arbitrary vertices from the shrunken colon to 
form a curve consisting of an ordered group of 3D points. This curve is generally similar to 
but may or may not coincide with the major axes, or the ultimately computed centerline. 
[0053] 3) Using the curve of the connected 3D points, we generate equally distanced 
planes perpendicular to the curve to define equal length segments along the shrunken colon. 
We map vertex points between these planes back to the original image of the colon to 
compute ring-like areas in the original colon. 

[0054] 4) Using the ring-like areas from the original colon, a centerline can be 
computed. 
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{0055J The indices of the vertices from the same segment are used to get a ring-like area 
"slice" from the original colon image. Any such ring-like area slice is processed to compute 
the local centerline point. It can be understood that the resulting points could be further used 
for computing better slices ("more" perpendicular on the centerline and more straight edged, 
5 smoothed or filtered centerline, etc). 

[0056] The original (not shrunken) colon is thus sliced. Our method could be used to 
process any similar shape (twisted cylinder). The method does not need the original CT 
slices and it uses only the reconstructed 3D surface. 

[0057] The results are based on colon versions with reduced-number of vertices. This 
10 permits us not only to reduce the total processing time but also to improve the algorithm 
performances. The shrunken colon model is accurate if the distances between points are 
much bigger (2-5 times) than the shrunken colon diameter. 

[0058] To perform the shrinking step, the distances between vertices are iteratively 
averaged, the result being a shrunken colon. For the purposes of this disclosure,, the two 

15 terms are equivalent; colon shrinking is the same as colon averaging. 

[0059] As will be explained later in this disclosure the "averaging" technique takes place in 
a three-dimensional basis. Referring to Figs. 2 A and 2B, a simplified explanation of the 
iterative averaging technique is offered in two dimensions utilizing for each averaging only 
two adjacent vertex points. While the reader will understand that the iterative averaging is 

20 three-dimensional and includes always more than two vertex points, the following examples 
are believed to be helpful. 

[0060] Referring to Fig. 2A, a circle 19 having vertex points 20 through 24 is illustrated. 
So-called "averaging" of the two-dimensional positions of the vertex points 20 through 24 is 
disclosed. By way of example, the averaging position of vertex point 21 can be considered. 

25 Specifically, the actual two-dimensional position of vertex point 21 is ignored. Instead, in 

accordance with the averaging technique here disclosed, the new two-dimensional position of 
vertex point 2V is the average of the respective positions of vertex point 20 and vertex point 
22. This results in the position of averaged vertex point 2V being on a chord midway 
between vertex point 20 and vertex point 22. 

30 [0061 ] Proceeding further with the two-dimensional analogy illustrated in Fig. 2A, this 

procedure is repeated for all vertex points located around circle 19. The result will be a new 
set of vertex points 20* through 24 1 . When these points are all connected with a smooth 
curve, a new, smaller and concentric circle 19' will be generated. 



WO 03/058553 PCT/US02/4064 1 

[0062] It can further be seen that what has been described constitutes only one iteration. 
By conducting many iterations, circle 19 can effectively be shrunken to a point. Thus, using 
this simplistic example, one can understand how the essentially cylindrical section of a 
human colon can be "shrunk." 
5 [0063] Referring to Fig. 2B, a straight line 29 is shown having vertex points 30 through 34 
along its length. The same averaging technique is applied. For example vertex point 31 has 
as its new position the average position of its vertex neighbors 30 and 32; this "new" position 
is labeled as vertex point 31\ It is immediately seen that the position of vertex point 31 and 
1 vertex point 3 1' is identical. 

1 0 [0064] Thus the reader will appreciate that line 29 does not shrink. 

[0065] In what follows, we will apply this illustrated two-dimensional technique to the 
three-dimensional image. Further, it will be understood that "average" positions will 
constitute three-dimensional average positions selected from many more than just two 
neighbors. Remembering that the serpentine colon is elongate along a major axis but 

1 5 generally irregularly cylindrical in shape, the reader can be given an intuitive understanding 
of why the iterative averaging process of this invention results in the shrunken colon having ' 
the same length but considerably reduced in diameter. 

[0066] It further will be remembered that each vertex point has at least two information 
sets. One of these information sets is a discrete identifier for each vertex point. This discrete 
20 identifier for each vertex point is not altered as a result of the shrinking process. The other of 
these information sets is the three-dimensional position of the vertex point. This information 
set is altered as a result of the averaging process. 

[0067] The disclosed "shrinking" causes all of the vertex points to move to and towards the 
major axis of the colon. 
25 [0068] Having given an oversimplified example of the averaging technique, a more 
technical description will be offered. 
[0069] The pseudo-code for one iteration is: 



Table 1 



STEP 


Description 


1) Take neighbors of Vi 




2) Average their coordinates. 


Get averageX, average Y, averageZ 


3) Assign Vi the above computed 
coordinates: 


Vi. X = averageX; Vi.Y = averageY; Vi.Z 
averageZ 
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[0070] All the operations from Table 1 are repeated for all the vertices of the surface for a 
certain number of times. Fig. 1 presents a 3D surface of the human colon. Fig. 3 presents the 
same surface shrunken 250 times. 

[0071] There are two important side effects of averaging: 
5 [0072] 1) Thinning of the whole shape; and, 

[0073] 2) Compression of the whole shape (its bounding box is smaller than the original 
shape). 

[0074] It is important to note the effect of thinning is more important than that of 
compression. 

1 0 [0075] Attention is directed to Fig. 4. In this example, the position information of a vertex 
point is altered utilizing the average positional information of its "neighbors." Thus, the 
alteration of the positional information of the vertex points can be referred to as "neighbors 
averaging." 

[0076] As shown in Fig. 4, the contribution of round neighbors 40, 41 (which are "at the 

1 5 same level" as the current vertex) to the vertex 47 movement is less significant than that of 
the other two vertices (displayed as squares 42, 43 and which are lower than current vertex). 
Observing Fig. 4, the section of the colon C illustrated is elongate (has its major axis 46) in 
the horizontal. The horizontal change in position due to neighbors averaging is not 
significant; a larger change of position is obtained perpendicular to the major axis, here on 

20 the vertical axis. 

[0077] If the iteration process is extremely long it is possible to obtain a straight line. 
[0078] The size of the movement depends actually on the curvature of the surface; points 
with greater curvature tend to move more. This is helpful because by shrinking we actually 
want to obtain a thin and twisted surface not a collapsed shape. The reader should understand 

25 that the shrunken colon is not used as the centerline of the normal colon. 

[0079] Referring to Figs. 5 A and 5B, it will be seen that the averaging technique of this 
invention does result in some shortening of the length of the colon. Points on the tight loops 
of the colon relative to the major axis present big curvatures. Thus points 53, 54, will tend to 
cause point 50 to move away from and even outside of the original surface of colon C. Points 

30 51, 52 will tend to cause point 50 to move toward the inside of the colon C. Other points on 
the outside of the curvature of colon C will move toward the inside of the colon. 
[0080] Table 2 shows some values of the colon bounding-box (normal and shrunken 
version): 

Table 2 
10 
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Colon 


tvne 


Xmin(cm) 


Ymin(cm) 


Zmin(cm) 


Xmax(cm) 


Ymax(cm) 


Zmax(cm) 


1 


normal 


9.94 


11.91 


4.00 


36.88 


32.59 


47.40 


thinned 


10.43 


13.86 


5.20 


35.89 


30.89 


44.22 


2 


normal 


8.81 


11.26 


3.98 


33.67 


34.16 


45.90 


thinned 


12.19 


14.43 


8.09 


30.16 


31.43 


41.94 



[0081] We can see that the variation of the bounding box is not as important as the thinning 
effect. For example, for colon 1, the "X" variation is: 

(35.89 -10.43) / (36.88 - 9.94) = 25.46 /26.94 = 94.5% 

[0082] The second step is to get a model of the shrunken colon. Our aim is to obtain a set 
of 3D points that should approximate the shrunken colon. These points will be used in the 
next step, when we want to get ring-like areas of the colon surface and use them to compute 
the centerline. 

[0083] We use this ordered set of 3D points to reduce the number of vertices processed. 
The ordered set of 3D points is spline interpolated and the control points are used to compute 
rings. We define a ring as a cylinder-like portion of the shrunken colon whose axis is more 
or less parallel with the colon model or colon centerline. 

[0084] A fundamental assumption for our algorithm can be stated: the shrunken colon 

centerline and its model are almost the same. This is true if the shrunken colon is thin and 

the distance between model points is bigger (2-5 times) than the shrunken colon diameter. 

[0085] We now model the shrunken colon by an ordered group of 3D points along the 

major axis of the colon in accordance with the second step of our procedure. 

[0086] Referring to Fig. 6, the algorithm to extract the points is quite simple: 

o We start with some random vertex point 60 located at the beginning of the 

shrunken colon. An easy area to select a random vertex point is the rectum 

because of its uniform location and more or less uniform direction with respect 

to any image of the colon. We select a first random vertex point on the rectum. 

The next random vertex point 61 selected is a vertex point that is located at a 

preselected distance D from the first random vertex point. This second vertex 
* 

point and the first vertex point define a three dimensional direction or vector 
V. We use this vector from the second vertex to choose the new (third) vertex 
point 62. 
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o From now on, every time we want to choose a new vertex we do the 
following: 

compute the new direction or vector (direction=current vertex — 
previous vertex); 

5 - define a search volume VC (see Fig. 6). This volume is the 

intersection between (see Figure 7A): 

the sphere 65 centered in last chosen vertex and having a radius 
equal to the largest preselected distance allowed between two 
consecutive vertices; 

1 0 • the sphere 66 centered in last chosen vertex and having a radius 

equal to the smallest preselected distance allowed between two 
consecutive vertices; and, 

the solid angle 67 central to the vector defined by the previous 
and current vertex. 

1 5 [0087] The solid angle could be even more than half a sphere. We use a large solid angle 
only to ensure the vertices are following the shrunken colon. If we do not use the vector V to 
define the spherical segment for search, it is possible to return back and select one of the 
previous vertex's close neighbors (or even the previous vertex itself). In such case it is 
possible to enter an infinite loop and never be able to end the colon processing. 

20 [0088] It will be understood that the solid angle should be as big as possible (even bigger 
than half a sphere) to ensure the proper track. 

[0089] Vertex choosing is random but nevertheless confined to a path generally along the 
major axis of the colon. The first vertex that happens to "be" within the correct search 
volume (VC) is chosen to be the next vertex. 
25 [00901 Our implementation uses typically a two dimensional search angle whose cosine 
minimum value is between (-0.2 ; 0.2). This means we are considering angle ranges from (- 
102°, +102°) to (-78°, +78°). See Table 3: min(cosine) >-l is the lowest possible value <S> 
maximum angle range: — 180° to 180°. 



Table 3 



Search Angle Values 


min(cosine) 


Range (degrees) 


Maximum range 


-1 


(-180°, +180°). 


Range 1 


-0.2 


(-102°, +102°) 


Range 2 


0.2 


(-78°, +78°) 
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Minimum range 



(-00.+0 0 ) 



[0091] The low limit value depends also on the shrunken colon diameter. In order to be 
able to follow every colon bend, we need the solid angle to be as large as possible (see Fig. 
8). If we take the maximum angle range, we risk tracking the colon incorrectly (and/or enter a 
5 infinite loop so that we'll never be able to finish data processing). 

[0092] In Fig. 7A, we displayed a situation when the value of 102° is not good (it is too 
big). The solid angle delimited spherical segment complies with our search conditions but is 
obviously not desired. Usually this situation doesn't happen because the ratio (distance 
between two vertices)/(shrunken colon diameter) is much larger. Ideally this ratio should be 
10 very large (in the range of 5-100), as we want the shrunken colon to be very thin. We have 
had success with the user selecting the value of the solid angle. For most colons, an angle of 
102° was sufficient. 

[0093] The last step is to use the shrunken colon model to compute the centerline of the 
real (not shrunken) colon. 
15 [0094] Reviewing the disclosure thus far, we have the original colon, its shrunken version 
and a model of it (an ordered set of 3D points or a curve). We need to determine an ordered 
set of points that should stay inside the colon. This set of ordered points should model the 
centerline of the colon like surface. For our purposes (polyps candidates discrimination in 
terms of their distance from origin), it is not necessary that the centerline be really "central" 
20 (provided this could be accurately defined). In general (for example to generate a "fly" path 
through virtual colon) if the centerline stays inside the surface of the colon, this is enough. 
[0095] With reference to Fig. 9, the main steps are: 
[0096] For each Pi of the model 

Take point Pi and Pi+1 of the model; 
25 - Compute the two planes 91, 92 perpendicular on the shrunken colon model 

(which model is actually a curve). The two planes are computed so that each 

of them passes through one of the corresponding points Pi and Pi+1. (see Fig. 

9); 

Select all the vertices between these two planes 91, 92 that belong to the 
30 shrunken colon; 
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Map these vertices to the original colon C. Thus and referring to Fig. 1 1 A, we 
get vertices that are grouped in ring-like areas R ring that are perpendicular on 
the major axis 100 of the shrunken colon; and, 

Process these vertices (for example average them or compute the center of 
5 their bounding box 101) (Figs 10 and 1 1). 

[0097} As a primary matter, it can be seen that we have developed a new method to 
"segment" cylindrical shapes of an irregular cylindrical surface, here the human colon. It can 
be used in general in image segmentation (2D, 3D, etc) and provides an easy and efficient 
way to process the colon. To get the centerline we can use any method to process the 

10 resulting slices. Further, we can obtain a point or a segment of line 102 to uniquely identify 
that slice. The resulting points or line segments define the desired centerline. Our needs 
require mapping distances determined back to the actual use of the colonoscope (polyps 
discrimination in terms of their distance from the rectum for example). 
[0098] We have run a software program based on the technique here displayed. This 

15 program was run on a Dell Optiplex GX lp PC with the features described in table 4: 



Table 4 



Processor 


Frequency 


Operating System 


Bus frequency 


Memory (RAM) 


Intel Pentium in 


550 MHz 


Windows NT 4.00 


100 MHz 


255 MB 



4.1 Shrinking 

[0099] Shrinking the colon is vital for the whole algorithm. Typically a colon surface has 
300000 — 500000 vertices. Averaging such a great number of vertices for 1000 times is 
20 very time consuming. For example if the number of vertices is 350000 and number of 

iterations is 1000 we have 1 1 seconds/iteration * 1000 iterations = 1 1000 seconds (almost 
180 minutes = 3 hours) only to shrink the colon. 

[0100] Also, due to the huge number of vertices, after 200-300 iterations, vertices are very 
close one to another. Further averaging does not generate appreciable three dimensional 
25 movement. This means that the effect of colon thinning is reduced (especially in the rectum 
and cecum — which are usually thick). The differences between 1000 iterations and 2000 
iterations are small. 

[0101] It will be appreciated that we do not need all the vertices supplied from an image of 
the colon shown in Fig. 1 to compute the shrunken colon. Therefore, before shrinking we 
30 preprocess the 3D colon surface by decimating the vertex points comprising the image of the 
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colon. For example, we prefer to reduce the number of vertices composing the surface by as 
much as 80%. 

[0102] Using a decimated colon image (65000 vertices instead on 350000), the average 
time per iteration is under 0.5 seconds (typically between 0.1 — 0.3 seconds), which leads to 
5 an average total time of 3 minutes per 1000 iterations. This time can be further improved by 
increasing the number of decimated vertices. If we are using 1/5 of the total number of points 
even the colon looks nice. 

4.2 Interpolation 

[0103J Since the shrunken colon is smooth, we are using spline functions to interpolate the 
10 first approximation. The interpolated points are then used to get the get the slices in the 

shrunken colon and in the original colon. Final points are again interpolated (using splines) 
in order to provide a smooth curve inside the colon. This last step (interpolation in normal 
colon) is not necessary — it was implemented only to provide "nicer" (smoother) curves. 
This could be useful, however, in virtual navigation — allowing a small image variation 
1 5 between two navigation points. 

4.3 Shrunken colon approximation 

[01041 The main parameters that control this stage of processing are (see Fig. 7B): 
vector of the solid angle for the next vertex; 

segment of the searched sphere delimited within the solid angle; and, 
20 - radius of the searched sphere. 

[01051 Typical values are summarized in table 5. 



Table 5 



Radius 


Width 


Angle (min value of cosine) 


0.7 * Az/20 


0.4 of radius 


-0.2 <min cos <0.2 



where: 

Az = Zmax — Zmin = amplitude of the whole colon on z axis (see figure 1 1 
25 and table 2); and, 

Radius is expressed as a fraction of Az/20 where 20 is an empirical value. 
[0106] The above value (Az/20) is chosen for the following reasons. In order to minimize 
data processing, we implemented a "locality" mechanism. We sort our vertices based on their 
Z coordinate (see figure 12). The total number of classes is 20 and each i-th class contain 
30 vertices whose Z coordinate is between i* Az/20 and (i+1)* Az/20 (+Z mm) where i = 0.. 19 
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(see figure 12). 20 is an empirical value and is based on the general features of the colon. If 
the processed shape is more twisted this value can be increased. Actually what matters is 
how tight are the loops of the shrunken shape: the tighter the loops are, bigger the number of 
classes. 

5 [0107] Referring to Fig. 12, classes are used to minimize the number of searches. For < 
every vertex in class i we search the vertices in class i, i-1, i+1. If the total number of classes 
is 20 we reduce our searches by almost 85% ((20-3)/20 = 17/20= 0.85 — provided each class 
has the same number of points). Of course the classes have different number of vertices (for 
example class 0 includes only a fraction of rectum vertices) but the principle is good and it 

10 could even be increased on the X axis (defining a rectangular grid). 

[01081 The shrunken colon is limited to a k*20 maximum number, where k is again 20 (and 
again empirical), which means no more than 400 points. These points are interpolated, 
introducing 3 new points between each 2 old points, and we obtain (400 - 1)*3 + 1 total 
numbers for the shrunken colon. These (almost) 1200 points generate 1200 points in the 

15 original colon, which in turn are again interpolated. Finally we have a centerline composed 
of maximum (1200-1) * 3 + 1 values = cca 3600. We never obtain such big values, typically 
the centerlines are modeled by less than 1000 points (500 - 900). 
[0109] The colon thickness (diameter) is also important. If this is not small the 
approximating line will be twisted and the slices will not be processed accurately. 

20 [01 1 0] One solution is to increase the shrinking (either the number of iterations or the 
number of decimated vertices). Another solution would be to provide an adaptive sphere 
radius, function of the colon diameter. This could be made either automatically (compute 
somehow the colon diameter) or using some a priori known information about colon shape 
(radius bigger at the beginning for the rectum and at the end for cecum, since these are 

25 usually the biggest segments of the colon). We are using the first solution in our algorithm. 
If, at the beginning, we were using 500 iterations to shrink, finally the value of 1000 proved 
to be enough for our needs. 

[0111] Another solution would be to increase the volume/surface averaged (considering not 
only the corresponding ring but also a part of the previous and next ring) in the original colon 
30 again on some a priori known areas where the colon diameter is usually big. We are using 
this actually on the whole colon in order to obtain a smoother centerline that models the 
colonoscope. 
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4.4 Computing slices in the original colon 

[0112] Using the points that approximate the shrunken colon is easy to determine slices 
(rings) as thick as we wish which are perpendicular on the shrunken colon centerline. 
Actually the main idea behind the shrunken colon is that the shrunken colon model and its 
5 centerline are almost the same. This is true because the shrunken colon is very thin and 
smooth, unlike the normal colon. 

[0113] Importantly, our method enables us to determine the vertices of the shrunken colon 
that belong to a certain slice (or ring) and then find their matching vertices in the original 
colon. These new vertices (from the original colon) form a ring that is more or less 
10 perpendicular on the colon centerline. The edges of this new ring (on the original colon) are 
not sharp (see Figs. 1 1 A and 1 IB) but we can take this ring as thick as we wish. 
[0114] All the values that right now are manually adjusted for each colon can be fully 
determined function of the local features of the colon (diameter). 

[0115] The whole process described above (excluding shrinking) takes less than 3 minutes 

15 (almost 2 minutes for a 65000 vertices colon). 

[0116] The computed slices can be further processed if we want an even more accurate 
centerline. Provided the slices are thin, a possible projection on a plane perpendicular on the 
above computed centerline can be used as illustrated in Fig. 13. In order to obtain an 
individual shape, we will need to re-index all the vertices. However such a 2D image can be 

20 processed easier to obtain a better approximation of the centerline. 

[0117] For example in Fig. 1 IB, we suggest taking the center of the bounding box while in 
our algorithm we use the average of all vertices (which may not always be inside the colon). 
In our program, using averaged thick cross-sections of the shrunken colon, we were able to 
obtain centerlines that reside completely inside the normal (not shrunken) colon. 

25 [0118] It will be understood that our algorithm uses only the 3D surface, it does not need 
the original CT images. Also it uses the coordinates as they are, as float numbers. The data 
does not need to be presented as a 3D rectilinear grid. This is one of the big differences 
comparing with other techniques. We are using vertices connectivity (vertex neighbors) and 
local vicinity. The resulting centerline is continuous and is interpolated using spline 

30 functions. 

[0119] The algorithm is "open" which means it can be improved using different techniques. 
The algorithm has basic steps and each one (or at least parts of them) could be changed and 
re-implemented using better approaches. For example, at some point we manage to get a 
cross-section of the colon. For each cross-section, a number of vertices form a "cylinder" 
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whose axis is on the colon centerline. We average all these vertices to compute one point on 
the centerline. 

[0120] It will be understood that we could use here any other method to compute the center 
line point: center of the bounding box, axis of the approximating cylinder computed using 
5 minimum of total square error, skeletons and so on. 

[0121] The technique is fully automated. All the parameters have empirically defined 
values and for normal colons user interaction is not necessary. However if the colons are 
collapsed (sometimes the CT segmentation module fails to determine the colon boundaries), 
or data acquisition has gaps, sometimes the shrunken colon modeling stops before the colon 
10 .end (before the cecum). In such cases we need user interaction (modifying search sphere 
radius and solid angle) in order to be able to cover the whole colon. 

[0122] The technique is relatively quick. Total average processing time is 5 minutes, from 
the original decimated surface to the ordered set of 3D points of the center-line, all the 
processing being done on a modest machine — Intel Pentium III PC, 550 MHz (see table 4). 

15 It can be further improved increasing the decimation degree for example. 

[0123] Another important advantage of our technique is that it already provides virtual fly- 
path elements (actually these elements are obtained before computing the centerline). The 
rings computed (slices in the original colon) are used to determine the centerline points but 
their projection on a plane perpendicular on their axis actually defines what the user wants to 

20 see during virtual navigation. 

[0124] Figure 1 IB shows also that, for virtual navigation purposes, the errors involved in 
centerline computation (if it stays or not inside the colon) don't affect the navigation 
accuracy. Using the slice's bounding box, we can provide an accurate and helpful image 
sequence (which is actually equivalent to considering the bounding box center as the 

25 centerline point). 

[0125] Our method is simple. All the steps presented above are only logically separated. 
Their implementation is quite simple. Although the complete program (written in C/C++) 
takes more than 1000 lines, the most important parts (shrunken colon modeling, vertices re- 
mapping etc) are simple functions. 
* 

30 [0126] Referring to Fig. 13, given a colon-like surface, we developed a method to obtain its 
centerline and to split the surface in slices perpendicular on its centerline. In Fig. 13 we show 
the colon centerline linearly disposed and how the parameters should be adapted to local 
conditions (in this case — local diameter) in order to increase the degree of automation of the 
whole process. 
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[0127J Centerline is a vital parameter for any CAD (Computer Assisted Diagnostic) tool. 
For example, a navigation path to display a 2D map of vertices' distances to the centerline 
could be generated. Observing asymmetries in the continuum of successive ring structures 
could improve polyp detection. 
5 [0128] Another application of out method is computing whole colon volume and local 
colon volume: the simplest way to do this would be to consider very thin slices, compute 
their area and compute local volume by multiplying each ring area by its height. Adding all 
these volumes (of all the colon rings) gives us the whole colon volume. 
[0129] Referring to Fig. 14, a random chosen selected modeling vertex point VO is 

1 0 designated along the colon length, the point here being adjacent one end of the shrunken 
colon image. The reader will understand that this image point could be chosen anywhere 
along the length of the shrunken colon image - it does not have to necessarily be adjacent one 
end of the colon (for example toward the cecum) or the other end of the colon (toward the 
rectum). (See Fig. 15) A process of neighbor identification and marking occurs. In the 

15 example here shown neighbors NvoO through Nvo5 are all identified and marked. Next, from 
each marked neighbor, identification and marking of previously unmarked neighbors occurs. 
Taking the example of neighbor vertex Nvo2, three neighboring and are marked vertices 
Nvo2_2, Nvo2_l, and Nvo2__0 are all marked. This process continues along the colon 
segment here illustrated. As the reader can understand, in the example here illustrated 

20 propagation of neighbor vertex identification and marking happens to proceed from the left of 
Fig. 14 to the right of Fig. 14. It could have just as well occurred in an opposite direction. It 
will be understood that once directionality of neighbor identification and marking of vertices 
is established, this directionality will occur from the arbitrarily selected random vertex point 
VO to an end of the colon. 

25 [0130] Referring to Fig. 15, the neighbor vertex identification and marking process 

continues until a located vertices exceeds a predetermined distance from the random vertex 
point VO initially designated. When this predetermined distance (plus or minus a tolerance) 
is reached, that vertex becomes a second designated modeling vertex Vl-1 . At this point, 
neighbor identification and marking will occur as illustrated in Fig. 14 until a third selected 

30 modeling vertex VI -2 is designated. This two-step process of neighbor identification and 

marking followed by designation of a modeling vertex will continue until an end of the colon 
is reached. 

[0131] Again referring to Fig. 15, the reader will understand that it is necessary to have the 
process of neighbor identification and marking followed by designation of a modeling vertex 
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in an opposite direction. This opposite direction will be from randomly designated modeling 
vertex point VO (see Figs 14 and 15) to the opposite end of the colon. To begin this process, 
once second selected modeling vertex Vl-1 is designated, an opposite modeling vertex V2-1 
is designated in a direction opposite to second selected modeling vertex Vl-1 at the 
predetermined distance. Once this choice is made, the two-step process of neighbor 
identification and marking followed by designation of modeling of vertices continues to the 
opposite end of the colon. 

[0132] Thus, it can be seen that the designation of modeling vertex points along the length 
of the colon effectively establishes a point trace of the shrunken colon image. This has the 
advantage of avoiding searches through solid angles of view for vertices as set forth in Fig. 6 
and is capable of following colon images having relatively sharp curvature. Further, the 
disclosed process is simple, iterative, and ideal for digital processing. 
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